Apparatus, system, and method of conference assistance

ABSTRACT

A conference assistance system and method for assisting a user in utilizing conference-related information, each of which: receives input of voice or writing of at least one meeting participant; determines whether text data representing the received voice or writing is to be used by the conference assistance system to generate assistance information, to generate a determination result; when the determination result indicates that the text data is to be used, generates, based on the text data, assistance information for assisting the meeting participant; and controls a display to display the generated assistance information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 16/192,983, filed on Nov. 16, 2018, which is a continuationapplication of U.S. application Ser. No. 15/670,130, filed Aug. 7, 2017,which claims priority pursuant to 35 U.S.C. § 119(a) to Japanese PatentApplication No. 2016-169663, filed on Aug. 31, 2016, in the Japan PatentOffice, the entire disclosure of each of which is hereby incorporated byreference herein.

BACKGROUND Technical Field

The present invention relates to an apparatus, system, and method ofconference assistance.

Description of the Related Art

Conference assistance systems for assisting a user at a conference areknown. Such conference assistance systems may be used, for example, toimprove effectiveness of organization management, operation planning,and decision-making of a committee or the like.

Further, at the conference, electronic whiteboards are often used by ameeting participant, to input user's statements. For example,handwritten characters and the like can be input to the electronicwhiteboard with an electronic pen or a user's finger on a touch panel.Further, the user's voices may be input to the electronic whiteboard.That is, the electronic whiteboard may be used to record contents ofdiscussion or a meeting agenda at the conference.

SUMMARY

Embodiments of the present invention include a conference assistancesystem and method for assisting a user in utilizing conference-relatedinformation, each of which: receives input of voice or writing of atleast one meeting participant; determines whether text data representingthe received voice or writing is to be used by the conference assistancesystem to generate assistance information, to generate a determinationresult; when the determination result indicates that the text data is tobe used, generates, based on the text data, assistance information forassisting the meeting participant; and controls a display to display thegenerated assistance information.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages and features thereof can be readily obtained and understoodfrom the following detailed description with reference to theaccompanying drawings, wherein:

FIG. 1 is a diagram illustrating a system configuration of a conferenceassistance system according to a first embodiment;

FIG. 2 is a diagram illustrating a hardware configuration of anelectronic whiteboard according to the first embodiment;

FIG. 3 is a diagram illustrating a hardware configuration of a characterconversion apparatus according to the first embodiment;

FIG. 4 is a diagram illustrating an exemplary functional configurationof the conference assistance system according to the first embodiment;

FIG. 5 is a sequence diagram illustrating an exemplary process ofinputting a handwritten character according to the first embodiment;

FIG. 6 is an illustration of exemplary transmission-permissioninformation;

FIG. 7 is an illustration of an exemplary screen on whichdecision-making assistance information is displayed;

FIG. 8 is an illustration of an exemplary screen on which an iconindicating non-transmission of contents of a conference is displayed;

FIG. 9 is a sequence diagram illustrating an exemplary process ofinputting voice according to the first embodiment;

FIG. 10 is an illustration of an exemplary screen on which informationindicating incapability of converting voice into a text is displayed;

FIG. 11 is a sequence diagram illustrating another exemplary process ofinputting voice according to the first embodiment;

FIG. 12 is a diagram illustrating an exemplary functional configurationof a conference assistance system according to a second embodiment;

FIG. 13 is a sequence diagram illustrating an exemplary process ofinputting a handwritten character according to the second embodiment;

FIG. 14 is an illustration of an exemplary screen for selecting whethercontents of a conference are to be transmitted;

FIG. 15 is a sequence diagram illustrating an exemplary process ofinputting voice according to the second embodiment;

FIG. 16 is a diagram illustrating an exemplary functional configurationof a conference assistance system according to a third embodiment;

FIG. 17 is a sequence diagram illustrating an exemplary process ofinputting a handwritten character according to the third embodiment;

FIG. 18 is an illustration of an exemplary screen for selecting whethertransmission of contents of a conference is to be suspended;

FIG. 19 is a sequence diagram illustrating an exemplary process ofinputting voice according to the third embodiment;

FIG. 20 is a diagram illustrating an exemplary functional configurationof a conference assistance system according to a fourth embodiment;

FIG. 21 is an illustration of exemplary confidential-word information;

FIG. 22 is a sequence diagram illustrating an exemplary process ofinputting a handwritten character according to the fourth embodiment;and

FIG. 23 is a sequence diagram illustrating an exemplary process ofinputting voice according to the fourth embodiment.

The accompanying drawings are intended to depict embodiments of thepresent invention and should not be interpreted to limit the scopethereof. The accompanying drawings are not to be considered as drawn toscale unless explicitly noted.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise.

In describing embodiments illustrated in the drawings, specificterminology is employed for the sake of clarity. However, the disclosureof this specification is not intended to be limited to the specificterminology so selected and it is to be understood that each specificelement includes all technical equivalents that have a similar function,operate in a similar manner, and achieve a similar result.

Now, embodiments of the present invention will be described in detailwith reference to the drawings.

<System Configuration>

First, a configuration of a conference assistance system 1 according toa first embodiment will be described with reference to FIG. 1. FIG. 1illustrates an exemplary system configuration of the conferenceassistance system 1 according to this embodiment.

As illustrated in FIG. 1, the conference assistance system 1 accordingto this embodiment includes an electronic whiteboard 10 and a characterconversion apparatus 20. In this embodiment, the electronic whiteboard10 and the character conversion apparatus 20 are provided at a usersite. The conference assistance system 1 further includes a voiceconversion service 30 and a decision-making assistance service 40, whichare connected to the electronic whiteboard 10 and the characterconversion apparatus 20 via a wide-area network N1, such as theInternet, in such a manner that the services 30 and 40 can communicatewith the voice conversion service 30 and the decision-making assistanceservice 40.

The electronic whiteboard 10 is used at, for example, a conference, aclassroom, or the like and can display data of an electronic file or thelike on a display and enables input of characters that are handwrittenwith a user's hand, an electronic pen, etc. and collecting (input) ofvoice with a microphone, for example. Such an electronic whiteboard 10is also referred to as an interactive whiteboard (IWB).

In addition, the electronic whiteboard 10 displays, on the display,various kinds of information (i.e., various kinds of information forassisting decision-making of a committee at a conference that uses theelectronic whiteboard 10, for example) provided from the decision-makingassistance service 40.

The character conversion apparatus 20 is an information processingapparatus that converts data representing handwritten characters(handwritten-character data) that have been input to the electronicwhiteboard 10 into text data by using, for example, the characterrecognition technology or the like.

Note that the electronic whiteboard 10 and the character conversionapparatus 20 are connected to each other so as to be capable ofcommunicating with each other through a network such as a local areanetwork (LAN) at an office. However, the electronic whiteboard 10 andthe character conversion apparatus 20 may be connected to each other soas to communicate with each other through the network N1, for example.

In this embodiment, the voice conversion service 30 is an externalsystem or apparatus, functioning as a server, that provides a servicefor converting data representing voice (voice data) that has been inputto the electronic whiteboard 10 into text data by using, for example,the voice recognition technology or the like.

The decision-making assistance service 40 is an external system orapparatus, functioning as a server, that provides a service forassisting decision-making of a user by using, for example, theartificial intelligence (AI) technology, an intelligent agent (IA), orthe like. That is, based on text data that has been received from thecharacter conversion apparatus 20, the voice conversion service 30, orthe like, the decision-making assistance service 40 provides variouskinds of information for assisting, for example, decision-making of acommittee to the electronic whiteboard 10.

Note that the voice conversion service 30 and the decision-makingassistance service 40 are each an external cloud service in thisembodiment. However, the voice conversion service 30 and thedecision-making assistance service 40 are not limited to this exampleand may be, for example, a service provided by an application serviceprovider (ASP), a web service, or the like.

In addition, the conference assistance system 1 illustrated in FIG. 1 isan example and may have another configuration. For example, thecharacter conversion apparatus 20 may be included in the electronicwhiteboard 10. In this case, the electronic whiteboard 10 convertshandwritten-character data into text data. In addition, for example, thevoice conversion service 30 may be included in the conference assistancesystem 1, or the character conversion apparatus 20 may be an externalapparatus that provides a cloud character-conversion service.

<Hardware Configuration>

Next, a hardware configuration of the electronic whiteboard 10 accordingto this embodiment will be described with reference to FIG. 2. FIG. 2illustrates an exemplary hardware configuration of the electronicwhiteboard 10 according to this embodiment.

As illustrated in FIG. 2, the electronic whiteboard 10 according to thisembodiment includes an input device 11, a display 12, an externalinterface (I/F) 13, a communication I/F 14, and a read only memory (ROM)15. The electronic whiteboard 10 according to this embodiment furtherincludes a random access memory (RAM) 16, a central processing unit(CPU) 17, a hard disk drive (HDD) 18, and a sound collecting device 19.These hardware components are connected to one another via a bus B1.

The input device 11 is a touch panel or the like and is used to input avariety of user operations (e.g., operation of inputting handwrittencharacters). The display 12 is a display device or the like and displaysvarious kinds of information (e.g., handwritten characters, varioustexts, and various icons). When the display 12 is provided externally,the display 12 is connected to the electronic whiteboard 10 via a cable,for example.

The external I/F 13 is an interface to an external device. An example ofthe external device is a recording medium 13 a. Thus, the electronicwhiteboard 10 can read or write data from or to the recording medium 13a or the like via the external I/F 13. Examples of the recording medium13 a include a universal serial bus (USB) memory, a compact disc (CD), adigital versatile disc (DVD), a secure digital (SD) memory card, and thelike.

The communication I/F 14 is an interface used to connect the electronicwhiteboard 10 to the network N1 or the like. Thus, the electronicwhiteboard 10 can communicate with another apparatus (e.g., thecharacter conversion apparatus 20 or the voice conversion service 30)via the communication I/F 14.

The HDD 18 is a nonvolatile memory device that stores programs and data.The programs and data stored in the HDD 18 are an operating system (OS),which is basic software that controls the whole electronic whiteboard10, an application program that provides a variety of functions on theOS, and the like.

The HDD 18 manages the stored programs and data by using file systemsand/or databases (DB). Note that the electronic whiteboard 10 mayinclude a driving device (e.g., solid state drive (SSD)) using a flashmemory as a memory medium instead of the HDD 18.

The ROM 15 is a nonvolatile semiconductor memory that is capable ofretaining programs and data even if the power is turned off. Theprograms and data stored in the ROM 15 are a basic input/output system(BIOS) executed when starting the electronic whiteboard 10, an OSsetting, a network setting, and the like. The RAM 16 is a volatilesemiconductor memory that temporarily retains the programs and data.

The CPU 17 is a processing device that controls the whole electronicwhiteboard 10 and that implements functions thereof by reading programsand data to the RAM 16 from memory devices such as the ROM 15 and theHDD 18 and by performing processes.

The sound collecting device 19 is a microphone, for example, andcollects sound around the electronic whiteboard 10.

The electronic whiteboard 10 according to this embodiment can performvarious processes, which will be described later, by having the hardwareconfiguration illustrated in FIG. 2.

Next, a hardware configuration of the character conversion apparatus 20according to this embodiment will be described with reference to FIG. 3.FIG. 3 illustrates an exemplary hardware configuration of the characterconversion apparatus 20 according to this embodiment.

As illustrated in FIG. 3, the character conversion apparatus 20according to this embodiment includes an input device 21, a display 22,an external I/F 23, and a communication I/F 24. The character conversionapparatus 20 according to this embodiment further includes a ROM 25, aRAM 26, a CPU 27, and an HDD 28. These hardware components are connectedto one another via a bus B2.

The input device 21 is a keyboard, a mouse, a touch panel, or the likeand is used to input a variety of user operations. The display 22 is adisplay device or the like and displays various kinds of information.Note that the character conversion apparatus 20 may establish aconnection between at least one of the input device 21 and the display22 and the bus B2, as needed.

The external I/F 23 is an interface to an external device. An example ofthe external device is a recording medium 23 a. Thus, the characterconversion apparatus 20 can read or write data from or to the recordingmedium 23 a or the like via the external I/F 23. Examples of therecording medium 23 a include a USB memory, a CD, a DVD, an SD memorycard, and the like.

The communication I/F 24 is an interface used to connect the characterconversion apparatus 20 to the network N1 or the like. Thus, thecharacter conversion apparatus 20 can communicate with another apparatus(e.g., the electronic whiteboard 10 or the decision-making assistanceservice 40) via the communication I/F 24.

The HDD 28 is a nonvolatile memory device that stores programs and data.The programs and data stored in the HDD 28 are an OS, which is basicsoftware that controls the whole character conversion apparatus 20, anapplication program that provides a variety of functions on the OS, andthe like.

The HDD 28 manages the stored programs and data by using predeterminedfile systems and/or databases (DB). Note that the character conversionapparatus 20 may include a driving device (e.g., solid state drive(SSD)) using a flash memory as a memory medium instead of the HDD 28.

The ROM 25 is a nonvolatile semiconductor memory that is capable ofretaining programs and data even if the power is turned off. Theprograms and data stored in the ROM 25 are, for example, a BIOS executedwhen starting the character conversion apparatus 20, an OS setting, anetwork setting, and the like. The RAM 26 is a volatile semiconductormemory that temporarily retains programs and data.

The CPU 27 is a processing device that controls the whole characterconversion apparatus 20 and that implements functions thereof by readingprograms and data to the RAM 26 from memory devices such as the ROM 25and the HDD 28 and by performing processes.

The character conversion apparatus 20 according to this embodiment canperform various processes, which will be described later, by having thehardware configuration illustrated in FIG. 3.

<Functional Configuration>

Next, a functional configuration of the conference assistance system 1according to this embodiment will be described with reference to FIG. 4.FIG. 4 is a diagram illustrating an exemplary functional configurationof the conference assistance system 1 according to this embodiment.

As illustrated in FIG. 4, the electronic whiteboard 10 according to thisembodiment includes an input receiver 101, a display control 102, anauthentication processor 103, a communicator 104, ahandwritten-character-data generation processor 105, a voice input 106,a voice-data generation processor 107, and a transmission permissiondeterminer 108. Each of these functional units is implemented by the CPU17 performing processes in accordance with one or more programsinstalled in the electronic whiteboard 10.

The electronic whiteboard 10 according to this embodiment furtherincludes a transmission-permission-information memory 109. This memorycan be implemented by using, for example, the HDD 18. Note that thememory may be implemented by using a memory device or the like connectedto the electronic whiteboard 10 via a network.

The input receiver 101 receives inputs of a variety of operationsperformed by a user. That is, the input receiver 101 receives inputs ofan operation of inputting handwritten characters, various selectionoperations, and the like performed by a user. In addition, the inputreceiver 101 receives an operation of inputting authenticationinformation for a user to start using the electronic whiteboard 10.

The display control 102 displays various kinds of information. That is,the display control 102 displays handwritten characters, a variety ofscreens, and the like. The display control 102 also displays textsrepresented by text data obtained through conversion performed by thevoice conversion service 30. The display control 102 further displaysdecision-making assisting information represented by decision-makingassistance data transmitted from the decision-making assistance service40.

The authentication processor 103 performs authentication based onauthentication information whose input has been received by the inputreceiver 101. That is, the authentication processor 103 performs anauthentication process by, for example, determining whether acombination of a user ID and a password (authentication information)whose input has been received by the input receiver 101 corresponds to acombination of a user ID and a password that is previously registered.

The communicator 104 communicates various kinds of data with anotherapparatus (e.g., the character conversion apparatus 20 or the voiceconversion service 30). That is, the communicator 104 transmits, forexample, handwritten-character data generated by thehandwritten-character-data generation processor 105 to the characterconversion apparatus 20. Thus, the electronic whiteboard 10 can convertthe handwritten-character data into text data by using the characterconversion apparatus 20 and can transmit the text data to thedecision-making assistance service 40.

The communicator 104 transmits, for example, voice data generated by thevoice-data generation processor 107 to the voice conversion service 30.Thus, the electronic whiteboard 10 can convert the voice data into textdata by using the voice conversion service 30 and can transmit the textdata to the decision-making assistance service 40.

Further, the communicator 104 receives decision-making assistance datafrom the decision-making assistance service 40 via the characterconversion apparatus 20 or the voice conversion service 30. Note thatthe decision-making assistance data is data representing information forassisting decision-making (e.g., information presenting a solution, analternative, or the like for an issue of an agenda of a committee).

Upon an operation of inputting a handwritten character being received bythe input receiver 101, the handwritten-character-data generationprocessor 105 generates data representing the handwritten character(handwritten-character data). In addition, in accordance with the resultof the determination performed by the transmission permission determiner108, the handwritten-character-data generation processor 105 transmitsthe generated handwritten-character data to the character conversionapparatus 20 via the communicator 104.

The voice input 106 inputs voice. That is, the voice input 106 inputssound (voice) collected by the sound collecting device 19.

The voice-data generation processor 107 generates data representingvoice (voice data) that has been input by the voice input 106. Inaddition, in accordance with the result of the determination performedby the transmission permission determiner 108, the voice-data generationprocessor 107 transmits the generated voice data to the voice conversionservice 30 via the communicator 104.

Referring to transmission-permission information stored in thetransmission-permission-information memory 109, the transmissionpermission determiner 108 determines, for example, whether contents of aconference (i.e., handwritten-character data and voice data) are to betransmitted to the character conversion apparatus 20 and the voiceconversion service 30.

In the transmission-permission information, as will be described later,a user ID of a user who uses the electronic whiteboard 10 andinformation for determining whether transmission is permitted (e.g.,appointment or the like of the user) are associated with each other.Accordingly, the transmission permission determiner 108 determineswhether transmission of the handwritten-character data and the voicedata is permitted in accordance with, for example, the appointment orthe like associated with the user ID of the user who has beenauthenticated by the authentication processor 103.

The transmission-permission-information memory 109 stores thetransmission-permission information that is used by the transmissionpermission determiner 108 to determine whether transmission of thehandwritten-character data and the voice data is permitted. Note thatdetails of the transmission-permission information will be describedlater.

As illustrated in FIG. 4, the character conversion apparatus 20according to this embodiment includes a communicator 201 and a characterconversion processor 202. Each of these functional units is implementedby the CPU 27 performing processes in accordance with one or moreprograms installed in the character conversion apparatus 20.

The communicator 201 communicates various kinds of data with anotherapparatus (e.g., the electronic whiteboard 10 or the decision-makingassistance service 40). That is, for example, the communicator 201transmits text data obtained through conversion performed by thecharacter conversion processor 202 to the decision-making assistanceservice 40. In addition, the communicator 201 transmits decision-makingassistance data received from the decision-making assistance service 40to the electronic whiteboard 10.

The character conversion processor 202 converts handwritten-characterdata received via the communicator 201 into text data. Note that thecharacter conversion processor 202 converts handwritten-character datainto text data by using, for example, the optical character recognition(OCR) technology or the like.

<Details of Processes>

Next, details of processes performed by the conference assistance system1 according to this embodiment will be described.

First, a case will be described with reference to FIG. 5 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, a participant at the conference inputs contents of theconference by handwriting. In this disclosure, the handwritingcorresponds to any writing made by the user using a user's finger, auser's hand, electronic pen, etc. FIG. 5 is a sequence illustrating anexemplary process of inputting a handwritten character according to thisembodiment.

First, the input receiver 101 of the electronic whiteboard 10 receivesan operation of inputting authentication information for a user, who isa participant at the conference, to start using the electronicwhiteboard 10 (step S501). Note that the operation of inputtingauthentication information may be, for example, performed by a userinputting a combination of a user ID and a password (authenticationinformation) on a screen displayed by the display control 102.

The operation of inputting authentication information may alternativelybe performed by a user, who brings an integrated circuit (IC) card closeinto the electronic whiteboard 10 at a predetermined position such aswhere a card reader is provided. In this case, the input receiver 101receives input of the authentication information that is recorded on theIC card.

Subsequently, the authentication processor 103 of the electronicwhiteboard 10 performs authentication based on the authenticationinformation whose input has been received by the input receiver 101(step S502). That is, the authentication processor 103 performs anauthentication process by, for example, determining whether acombination of a user ID and a password (authentication information)whose input has been received by the input receiver 101 corresponds to acombination of a user ID and a password that is previously registered ina memory such as the transmission-permission information memory 109.Alternatively, the authentication processor 103 authenticates the user,when the user ID read out from the IC card of the user is found in thetransmission-permission information stored in thetransmission-permission-information memory 109.

Alternatively, the electronic whiteboard 10 may send authenticationinformation input by the user to an authentication server, which storesauthentication information of the registered user, to request for userauthentication. In such case, a part of the transmission permissiondeterminer 108 that performs authentication and thetransmission-permission-information memory 109 do not have to beprovided at the electronic whiteboard 10.

The following description is made on the assumption that theauthentication in step S502 above was successful. Note that theauthentication process in step S502 above is not necessarily performed.That is, the electronic whiteboard 10 according to this embodiment atleast receives input of the user ID in step S501 above.

Subsequently, referring to transmission-permission information stored inthe transmission-permission-information memory 109, the transmissionpermission determiner 108 of the electronic whiteboard 10 determineswhether handwritten-character data is to be transmitted (whethertransmission is permitted) to the character conversion apparatus 20(step S503).

The transmission-permission information stored in thetransmission-permission-information memory 109 will now be describedwith reference to FIG. 6. FIG. 6 illustrates exemplarytransmission-permission information stored in thetransmission-permission-information memory 109.

As illustrated in FIG. 6, the transmission-permission information storedin the transmission-permission-information memory 109 includes “userID”, “name”, “appointment”, and the like as data items. “User ID” is anidentifier for identifying each user who uses the electronic whiteboard10. “Name” is the name of the user. “Appointment” is the appointment(e.g., department manager or division chief) of the user. In thismanner, the user ID and information about the appointment and the likeare associated with each other in the transmission-permissioninformation. The transmission-permission information may further includea password of the user in association with the user ID.

The transmission permission determiner 108 determines whethertransmission is permitted by determining, for example, whether theappointment associated with the user ID included in the authenticationinformation whose input has been received by the input receiver 101 is apredetermined appointment.

That is, referring to the transmission-permission information stored inthe transmission-permission-information memory 109, if the appointmentassociated with the user ID included in the authentication informationis, for example, a department manager or a higher appointment, thetransmission permission determiner 108 determines that transmission isrestricted. On the other hand, if the appointment associated with theuser ID included in the authentication information is, for example,lower than the department manager, the transmission permissiondeterminer 108 determines that transmission is permitted.

Note that if a plurality of pieces of authentication information areinput (e.g., if all participants at a conference swipe their IC cards)in step S501 above, the transmission permission determiner 108 maydetermine whether an appointment associated with at least one of theuser IDs included in the plurality of pieces of authenticationinformation is a predetermined appointment.

Alternatively, the transmission permission determiner 108 may determinewhether an appointment associated with the user ID having the highestlevel of appointment, is higher than the predetermined appointment.

In addition to the determination as to whether transmission is permittedin accordance with the appointment of each user, the transmissionpermission determiner 108 may determine whether transmission ispermitted in accordance with, for example, the organization, department,team, division, or the like to which the user belongs. Furthermore, thetransmission permission determiner 108 may determine whethertransmission is permitted in accordance with, for example, the deviceID, installed location, or the like of the electronic whiteboard 10instead of the user who uses the electronic whiteboard 10. Note that“name”, “appointment”, “organization”, “department”, “team”, “division”,and the like are examples of the attribute information of a user in thisexample. In addition, “appointment” is an example of the appointmentinformation in this example.

If the transmission permission determiner 108 determines thattransmission is permitted in step S503, the conference assistance system1 according to this embodiment performs step S504 through step S509,which will be described later, every time an operation of inputting ahandwritten character is performed in the electronic whiteboard 10.

That is, the input receiver 101 of the electronic whiteboard 10 receivesan operation of inputting a handwritten character performed by a user.Subsequently, the handwritten-character-data generation processor 105 ofthe electronic whiteboard 10 generates data representing the handwrittencharacter (handwritten-character data) received by the input receiver101 (step S504).

At this time, the handwritten-character-data generation processor 105may generate image data representing each handwritten character that hasbeen input by a user as the handwritten-character data or may generatestroke data representing each stroke of each handwritten character asthe handwritten-character data. Note that the operation of inputting ahandwritten character may be performed by a user by, for example,writing the character on the display 12 with an electronic pen or auser's finger.

The display control 102 of the electronic whiteboard 10 displays thehandwritten character represented by the handwritten-character datagenerated by the handwritten-character-data generation processor 105 onthe display 12 (step S505). Thus, the handwritten character that hasbeen input by a user is displayed.

Subsequently, the handwritten-character-data generation processor 105 ofthe electronic whiteboard 10 transmits the handwritten-character data tothe character conversion apparatus 20 via the communicator 104 (stepS506).

Upon reception of the handwritten-character data via the communicator201, the character conversion processor 202 of the character conversionapparatus 20 converts the handwritten-character data into text data(step S507).

Subsequently, the character conversion processor 202 of the characterconversion apparatus 20 transmits the text data obtained throughconversion in step S507 above to the decision-making assistance service40 via the communicator 201 (step S508). Note that transmission of thetext data can be performed through, for example, a web applicationprogramming interface (WebAPI) provided by the decision-makingassistance service 40.

In response, the decision-making assistance service 40 transmitsdecision-making assistance data to the electronic whiteboard 10 via thecharacter conversion apparatus 20.

Note that the character conversion apparatus 20 may transmit the textdata back to the electronic whiteboard 10 in step S508 above. In thiscase, the electronic whiteboard 10 transmits the text data transmittedback from the character conversion apparatus 20 to the decision-makingassistance service 40.

Upon reception of the decision-making assistance data via thecommunicator 104, the display control 102 of the electronic whiteboard10 displays decision-making assistance information (step S509).

That is, as illustrated in FIG. 7, for example, the display control 102displays decision-making assistance information G110 represented by thedecision-making assistance data on a screen G100 displayed on thedisplay 12.

Note that in the example illustrated in FIG. 7, the displayeddecision-making assistance information G110 indicates that Plan B, whichis similar to handwritten characters “Plan A” that have been input by auser, is being developed by another project (Project X).

In the above manner, with the conference assistance system 1 accordingto this embodiment, the handwritten character that has been input by auser is converted into a text, and then the text is transmitted to thedecision-making assistance service 40, and thereby, for example,information for assisting decision-making of a committee or the like isdisplayed on the electronic whiteboard 10. Thus, a committee that usesthe electronic whiteboard 10 can expect a reduction in fruitlessarguments, prompt decision-making, and the like.

On the other hand, if the transmission permission determiner 108determines that the transmission is restricted in step S503, theconference assistance system 1 according to this embodiment performsstep S510, which will be described later. In addition, after step S510,the conference assistance system 1 according to this embodiment performsstep S511 and step S512, which will be described later, every time anoperation of inputting a handwritten character is performed in theelectronic whiteboard 10.

That is, the display control 102 of the electronic whiteboard 10displays an icon indicating that contents of a conference are not to betransmitted (i.e., indicating that handwritten-character data is not tobe transmitted to the character conversion apparatus 20) (step S510).

That is, the display control 102 displays, for example, an icon G210indicating that contents of a conference are not to be transmitted(e.g., contents of a conference are confidential) on a screen G200displayed on the display 12, as illustrated in FIG. 8. Thus, forexample, participants at the conference can understand that the contentsthat have been input to the electronic whiteboard 10 by handwriting arenot to be transmitted externally (to the decision-making assistanceservice 40). Accordingly, for example, the participants at theconference can feel free to argue over an agenda or the like includingconfidential information or the like. Note that the icon G210 is anexample of the display component in the scope of claims.

Note that on the screen G200 illustrated in FIG. 8, for example, if auser depresses the icon G210 (unlocks “confidential”), the electronicwhiteboard 10 may transmit contents of the conference (i.e., transmithandwritten-character data to the character conversion apparatus 20).

Subsequently, the input receiver 101 of the electronic whiteboard 10receives an operation of inputting a handwritten character performed bya user. Subsequently, the handwritten-character-data generationprocessor 105 of the electronic whiteboard 10 generates datarepresenting the handwritten character (handwritten-character data)received by the input receiver 101 (step S511).

Subsequently, the display control 102 of the electronic whiteboard 10displays the handwritten character represented by thehandwritten-character data generated by the handwritten-character-datageneration processor 105 on the display 12 (step S512). Thus, thehandwritten character that has been input by a user is displayed.

In the above manner, the electronic whiteboard 10 according to thisembodiment determines whether contents of the conference, which havebeen input by handwriting, are to be transmitted externally (to thedecision-making assistance service 40 or the like) in accordance withthe appointment or the like of a participant (user) who uses theelectronic whiteboard 10.

Thus, with the conference assistance system 1 according to thisembodiment, for example, at a conference in which a director or the likeof a company participates, among contents of the conference that havebeen input to the electronic whiteboard 10 by handwriting, confidentialinformation can be prevented from being leaked to the decision-makingassistance service 40, even when such service 40 is provided on a publicnetwork such as the Internet.

There is also a case in which the decision-making assistance service 40provides optimal decision-making assistance data from accumulated piecesof information of the past. With such a decision-making assistanceservice 40, confidential information (or information based on theconfidential information) transmitted to the decision-making assistanceservice 40 might possibly be provided to another user of the electronicwhiteboard 10 as decision-making assistance information.

Thus, with the conference assistance system 1 according to thisembodiment, confidential information or the like can be prevented frombeing provided to another user of the electronic whiteboard 10 asdecision-making assistance information.

Next, a case will be described with reference to FIG. 9 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, voice of a speech of a participant or the like at theconference collected as contents of the conference is input. FIG. 9 is asequence illustrating an exemplary process of inputting voice accordingto this embodiment. Note that step S901 through step S903 aresubstantially the same as step S501 through step S503 in FIG. 5,respectively, and therefore description thereof will be omitted.

If the transmission permission determiner 108 determines thattransmission is permitted in step S903, the conference assistance system1 according to this embodiment performs step S904 through step S908described below every time voice is collected by the sound collectingdevice 19 of the electronic whiteboard 10.

That is, the voice input 106 of the electronic whiteboard 10 inputs thevoice collected by the sound collecting device 19. Subsequently, thevoice-data generation processor 107 of the electronic whiteboard 10generates data representing the voice (voice data) that has been inputby the voice input 106 (step S904).

Subsequently, the voice-data generation processor 107 of the electronicwhiteboard 10 transmits the voice data to the voice conversion service30 via the communicator 104 (step S905). In response, the voiceconversion service 30 converts the voice data into text data andtransmits the text data back to the electronic whiteboard 10.

Subsequently, the display control 102 of the electronic whiteboard 10displays a text represented by the text data transmitted back from thevoice conversion service 30 (step S906). Thus, contents or the like of aspeech of a participant at the conference are displayed as a text on thedisplay 12 of the electronic whiteboard 10. Note that if, for example, aplurality of pieces of text data and the reliability of the text data(index representing likeliness of conversion) are transmitted back fromthe voice conversion service 30, the display control 102 may display atext represented by the most reliable text data.

On the other hand, the voice conversion service 30 transmits the textdata to the decision-making assistance service 40 (step S907). Inresponse, the decision-making assistance service 40 transmitsdecision-making assistance data to the electronic whiteboard 10 via thevoice conversion service 30.

Note that if the text data is not transmitted in step S907 above, theelectronic whiteboard 10 may transmit the text data to thedecision-making assistance service 40.

Upon reception of the decision-making assistance data via thecommunicator 104, the display control 102 of the electronic whiteboard10 displays decision-making assistance information (step S908). Thus, asin step S509 in FIG. 5, the decision-making assistance information isdisplayed on the display 12 of the electronic whiteboard 10.

On the other hand, if the transmission permission determiner 108determines that transmission is restricted in step S903, the conferenceassistance system 1 according to this embodiment performs step S909described below.

That is, the display control 102 of the electronic whiteboard 10displays, for example, a screen G300 illustrated in FIG. 10 (step S909).The screen G300 illustrated in FIG. 10 is a screen that displaysinformation indicating that conversion from voice into a text is notallowed.

In the above manner, if the transmission permission determiner 108determines that transmission is restricted, the electronic whiteboard 10according to this embodiment notifies a user that converting voice intoa text is not allowed.

As described above, the conference assistance system 1 according to thisembodiment determines whether handwritten-character data or voice datais to be transmitted in accordance with a user who uses the electronicwhiteboard 10. Thus, the conference assistance system 1 according tothis embodiment can prevent, for example, contents of the conferencefrom being transmitted in accordance with the attribute (e.g.,appointment or department) of a user who uses the electronic whiteboard10.

Thus, with the conference assistance system 1 according to thisembodiment, for example, it is possible to prevent leakage (to thedecision-making assistance service 40 or the like) of contents ofspeeches, handwritten contents, and the like at a highly confidentialconference or the like in which a director or the like of a companyparticipates.

Although the input of a handwritten character and the input of voicehave been described above with reference to different sequences, theseinputs may be performed at the same time. That is, voice and handwrittencharacter may be input to the electronic whiteboard 10 according to thisembodiment at the same time.

Note that permission of transmission to the decision-making assistanceservice 40 can be controlled in some cases by transmitting, to the voiceconversion service 30, a flag (transmission-permission flag) indicatingwhether transmission of text data to the decision-making assistanceservice 40 is permitted in addition to voice data.

Thus, a process of inputting voice in a case in which the electronicwhiteboard 10 transmits a transmission-permission flag in addition tovoice data will be described below with reference to FIG. 11. FIG. 11 isa sequence illustrating another exemplary process of inputting voiceaccording to this embodiment. Note that step S1101 through step S1103are substantially the same as step S901 through step S903 in FIG. 9,respectively, and therefore description thereof will be omitted.

If the transmission permission determiner 108 determines thattransmission is permitted in step S1103, the conference assistancesystem 1 according to this embodiment performs step S1104 through S1108described below every time voice is collected by the sound collectingdevice 19 of the electronic whiteboard 10.

That is, the voice input 106 of the electronic whiteboard 10 inputs thevoice collected by the sound collecting device 19. Subsequently, thevoice-data generation processor 107 of the electronic whiteboard 10generates data representing the voice (voice data) that has been inputby the voice input 106 (step S1104).

Subsequently, the voice-data generation processor 107 of the electronicwhiteboard 10 transmits the voice data and the transmission-permissionflag to the voice conversion service 30 via the communicator 104 (stepS1105). At this time, the voice-data generation processor 107 sets thevalue of the transmission-permission flag to “1” indicating thattransmission of text data to the decision-making assistance service 40is permitted and transmits the transmission-permission flag.

In response, the voice conversion service 30 converts the voice datainto text data and transmits the text data back to the electronicwhiteboard 10.

Subsequently, the display control 102 of the electronic whiteboard 10displays a text represented by the text data that has been transmittedback from the voice conversion service 30 (step S1106).

On the other hand, since the value of the transmission-permission flagis “1”, the voice conversion service 30 transmits the text data to thedecision-making assistance service 40 (step S1107). In response, thedecision-making assistance service 40 transmits decision-makingassistance data to the electronic whiteboard 10 via the voice conversionservice 30.

Upon reception of the decision-making assistance data via thecommunicator 104, the display control 102 of the electronic whiteboard10 displays decision-making assistance information (step S1108).

On the other hand, if the transmission permission determiner 108determines that transmission is restricted in step S1103, the conferenceassistance system 1 according to this embodiment performs step S1109through step S1111 described below.

That is, as in step S1104 above, the voice input 106 of the electronicwhiteboard 10 inputs the voice that has been collected by the soundcollecting device 19. Subsequently, the voice-data generation processor107 of the electronic whiteboard 10 generates data representing thevoice (voice data) that has been input by the voice input 106 (stepS1109).

Subsequently, the voice-data generation processor 107 of the electronicwhiteboard 10 transmits the voice data and the transmission-permissionflag to the voice conversion service 30 via the communicator 104 (stepS1110). At this time, the voice-data generation processor 107 sets thevalue of the transmission-permission flag to “0” indicating thattransmission of text data to the decision-making assistance service 40is restricted and transmits the transmission-permission flag.

In response, the voice conversion service 30 converts the voice datainto text data and transmits the text data back to the electronicwhiteboard 10. Note that since the value of the transmission-permissionflag is “0”, the voice conversion service 30 does not transmit the textdata to the decision-making assistance service 40.

Subsequently, as in step S1106 above, the display control 102 of theelectronic whiteboard 10 displays a text represented by the text datathat has been transmitted back from the voice conversion service 30(step S1111).

In the above manner, if transmission of text data to the decision-makingassistance service 40 can be controlled by transmitting atransmission-permission flag to the voice conversion service 30, theelectronic whiteboard 10 according to this embodiment transmits thetransmission-permission flag in addition to voice data to the voiceconversion service 30. Thus, the electronic whiteboard 10 according tothis embodiment can display text data obtained through conversion of thevoice data even if the transmission permission determiner 108 determinesthat transmission is restricted, as described in step S1109 through stepS1111 above.

Next, a second embodiment will be described. The second embodiment willdescribe a case in which, for example, before starting a conference, auser can select in the electronic whiteboard 10 whether contents of theconference are to be transmitted.

<Functional Configuration>

First, a functional configuration of the conference assistance system 1according to this embodiment will be described with reference to FIG.12. FIG. 12 illustrates an exemplary functional configuration of theconference assistance system 1 according to this embodiment. Note thatfunctional components that are substantially the same as those in thefirst embodiment will not be repeatedly described.

As illustrated in FIG. 12, the electronic whiteboard 10 of theconference assistance system 1 according to this embodiment includes atransmission permission determiner 108A. In accordance with selection ona predetermined screen (screen for a user to select whether transmissionof contents of a conference is permitted) displayed by the displaycontrol 102, the transmission permission determiner 108A determineswhether handwritten-character data and voice data are to be transmittedto the character conversion apparatus 20 and the voice conversionservice 30.

<Details of Processes>

Next, details of processes performed by the conference assistance system1 according to this embodiment will be described.

First, a case will be described with reference to FIG. 13 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, a participant or the like at the conference inputs contents ofthe conference by handwriting. FIG. 13 is a sequence illustrating anexemplary process of inputting a handwritten character according to thisembodiment. Note that step S1301 and step S1302 are substantially thesame as step S501 and step S502 in FIG. 5, respectively, and thereforedescription thereof will be omitted.

Following step S1302, the display control 102 of the electronicwhiteboard 10 displays a screen G400 for a user to select whethercontents of the conference are to be transmitted, such as the screenillustrated in FIG. 14 (step S1303).

Note that the screen G400 illustrated in FIG. 14 includes an “YES”button G410 for selecting transmission of contents of the conference(transmission permission) and a “NO” button G420 for selectingnon-transmission of contents of the conference (transmissionrestriction).

Subsequently, the transmission permission determiner 108A of theelectronic whiteboard 10 determines whether handwritten-character datais to be transmitted (transmission permission/restriction) to thecharacter conversion apparatus 20 (step S1304). That is, thetransmission permission determiner 108A determines, for example, whichof the “YES” button G410 and the “NO” button G420 has been selected onthe screen G400 illustrated in FIG. 14 to determine transmissionpermission/restriction.

If the transmission permission determiner 108A determines that thetransmission is permitted in step S1304, the conference assistancesystem 1 according to this embodiment performs step S1305 through stepS1310 every time an operation of inputting a handwritten character isperformed in the electronic whiteboard 10. Note that step S1305 throughstep S1310 are substantially the same as step S504 through step S509 inFIG. 5, respectively, and therefore description thereof will be omitted.

On the other hand, if the transmission permission determiner 108Adetermines that the transmission is restricted in step S1304, theconference assistance system 1 according to this embodiment performsstep S1311. Subsequently, the conference assistance system 1 accordingto this embodiment performs step S1312 and step S1313 every time anoperation of inputting a handwritten character is performed in theelectronic whiteboard 10. Note that step S1311 through step S1313 aresubstantially the same as step S510 through S512 in FIG. 5,respectively, and therefore description thereof will be omitted.

In the above manner, with the conference assistance system 1 accordingto this embodiment, a user selects whether the transmission is permittedin the electronic whiteboard 10 so as to control whether transmission ofthe handwritten-character data is permitted. Thus, before starting aconference that uses the electronic whiteboard 10 according to thisembodiment, for example, a participant or the like at the conference canselect whether transmission of the handwritten-character data ispermitted in accordance with the agenda to be argued about.

Next, a case will be described with reference to FIG. 15 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, voice of a speech of a participant or the like at theconference collected as contents of the conference is input. FIG. 15 isa sequence illustrating an exemplary process of inputting voiceaccording to this embodiment.

Note that step S1501 and step S1502 are substantially the same as stepS901 and step S902 in FIG. 9, respectively, and therefore descriptionthereof will be omitted.

Following step S1502, as in step S1303 above, the display control 102 ofthe electronic whiteboard 10 displays the screen G400 for a user toselect whether contents of the conference are to be transmitted, such asthe screen illustrated in FIG. 14 (step S1503).

Subsequently, the transmission permission determiner 108A of theelectronic whiteboard 10 determines whether voice data is to betransmitted (transmission permission/restriction) to the voiceconversion service 30 (step S1504). That is, the transmission permissiondeterminer 108A determines, for example, which of the “YES” button G410and the “NO” button G420 has been selected on the screen G400illustrated in FIG. 14 to determine transmission permission/restriction.

If the transmission permission determiner 108A determines that thetransmission is permitted in step S1504, the conference assistancesystem 1 according to this embodiment performs step S1505 through stepS1509 every time voice is collected by the sound collecting device 19 ofthe electronic whiteboard 10. Note that step S1505 through step S1509are substantially the same as step S904 through step S908 in FIG. 9,respectively, and therefore description thereof will be omitted.

On the other hand, if the transmission permission determiner 108Adetermines that the transmission is restricted in step S1504, theconference assistance system 1 according to this embodiment performsstep S1510. Note that step S1510 is substantially the same as step S909in FIG. 9, and therefore description thereof will be omitted.

In the above manner, with the conference assistance system 1 accordingto this embodiment, a user selects whether the transmission is permittedin the electronic whiteboard 10 so as to control transmission of thevoice data is permitted. Thus, before starting a conference that usesthe electronic whiteboard 10 according to this embodiment, for example,a participant or the like at the conference can select whether thetransmission of the voice data is permitted in accordance with theagenda to be argued about.

As described above, with the conference assistance system 1 according tothis embodiment, for example, before starting a conference, a user ofthe electronic whiteboard 10 can select whether contents of theconference are to be transmitted to the decision-making assistanceservice 40. Thus, with the conference assistance system 1 according tothis embodiment, for example, it is possible to select whether contentsof the conference are to be transmitted to the decision-makingassistance service 40 in accordance with the agenda or the like of theconference to be held.

Next, a third embodiment will be described. In the third embodiment, iftext data includes a term (confidential word) that has been registeredin advance, for example, or a term (similar word) that is similar to theterm, the decision-making assistance service 40 transmits backdecision-making assistance data indicating that contents of a conferenceinclude confidential information.

Thus, for example, upon reception of the decision-making assistance dataindicating that contents of the conference include confidentialinformation during the conference, the electronic whiteboard 10according to this embodiment displays a predetermined screen so as toenable a user to select whether future transmission of contents of theconference is to be suspended.

<Functional Configuration>

First, a functional configuration of the conference assistance system 1according to this embodiment will be described with reference to FIG.16. FIG. 16 illustrates an exemplary functional configuration of theconference assistance system 1 according to this embodiment. Note thatfunctional components that are substantially the same as those in thefirst embodiment will not be repeatedly described.

As illustrated in FIG. 16, the electronic whiteboard 10 of theconference assistance system 1 according to this embodiment includes adecision-making-assistance-data determiner 110. In addition, theelectronic whiteboard 10 of the conference assistance system 1 accordingto this embodiment does not include the transmission permissiondeterminer 108. The decision-making-assistance-data determiner 110determines whether decision-making assistance data that has beenreceived via the communicator 104 indicates that contents of aconference include confidential information.

<Details of Processes>

Next, details of processes performed by the conference assistance system1 according to this embodiment will be described.

First, a case will be described with reference to FIG. 17 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, a participant or the like at the conference inputs contents ofthe conference by handwriting. FIG. 17 is a sequence illustrating anexemplary process of inputting a handwritten character according to thisembodiment. Note that step S1701 and step S1702 are substantially thesame as step S1301 and step S1302 in FIG. 13, respectively, andtherefore description thereof will be omitted.

The conference assistance system 1 according to this embodiment performsstep S1703 through step S1710 every time an operation of inputting ahandwritten character is performed in the electronic whiteboard 10. Notethat step S1703 through step S1706 are substantially the same as stepS1305 through S1308 in FIG. 13, respectively, and therefore descriptionthereof will be omitted.

Following step S1706, text data obtained through conversion in stepS1706 is transmitted to the decision-making assistance service 40 viathe communicator 201 of the character conversion apparatus 20 (stepS1707).

In response, the decision-making assistance service 40 transmitsdecision-making assistance data to the electronic whiteboard 10 via thecharacter conversion apparatus 20. At this time, the decision-makingassistance service 40 determines, for example, a text represented by thetext data received from the character conversion apparatus 20 includes aconfidential word or a similar word. If the decision-making assistanceservice 40 determines that the text represented by the text dataincludes a confidential word or a similar word, the decision-makingassistance service 40 transmits decision-making assistance dataindicating that confidential information is included to the electronicwhiteboard 10 via the character conversion apparatus 20.

Note that the confidential word herein is a term that has beenregistered in the decision-making assistance service 40 in advance by auser or the like of the electronic whiteboard 10 and is a term relatedto confidential matter such as “lawsuit”, “management planning”, or“patent”. In addition, the similar word is a term that has a meaningsimilar to or related to the meaning of the confidential word and is aterm similar to a confidential word “lawsuit”, such as “trial”,“sentence”, or “suit”. The similar word is, for example, determined bythe decision-making assistance service 40 by using the AI technology oran IA based on the confidential word registered by a user or the like,using such as a dictionary.

The decision-making-assistance-data determiner 110 of the electronicwhiteboard 10 determines whether the decision-making assistance datareceived via the communicator 104 indicates that confidentialinformation is included (step S1708).

If it is determined in step S1708 that the decision-making assistancedata indicates that confidential information is included, the displaycontrol 102 displays a screen G500 for selecting suspension oftransmission of contents of the conference, such as the screenillustrated in FIG. 18 (step S1709). Note that the screen G500illustrated in FIG. 18 is an example of the selection screen in thescope of claims.

Note that the screen G500 illustrated in FIG. 18 includes an “YES”button G510 for selecting suspension of transmission of contents of theconference and a “NO” button G520 for selecting non-suspension oftransmission of contents of the conference.

On the other hand, if it is determined in step S1708 that thedecision-making assistance data indicates that confidential informationis not included, the display control 102 displays decision-makingassistance information (step S1710).

In the above manner, with the conference assistance system 1 accordingto this embodiment, for example, a user of the electronic whiteboard 10depresses the “YES” button G510 on the screen G500 illustrated in FIG.18 to refrain from future transmission of contents of the conference(handwritten-character data). On the other hand, with the conferenceassistance system 1 according to this embodiment, for example, a user ofthe electronic whiteboard 10 depresses the “NO” button G520 on thescreen G500 illustrated in FIG. 18 to continue transmission of contentsof the conference (handwritten-character data).

Next, a case will be described with reference to FIG. 19 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, voice of a speech of a participant or the like at theconference collected as contents of the conference is input. FIG. 19 isa sequence illustrating an exemplary process of inputting voiceaccording to this embodiment. Note that step S1901 and step S1902 aresubstantially the same as step S1501 and step S1502 in FIG. 15,respectively, and therefore description thereof will be omitted.

The conference assistance system 1 according to this embodiment performsstep S1903 through step S1909 every time voice is collected by the soundcollecting device 19 of the electronic whiteboard 10. Note that stepS1903 through step S1905 are substantially the same as step S1505through step S1507 in FIG. 15, respectively, and therefore descriptionthereof will be omitted.

The voice conversion service 30 transmits, to the decision-makingassistance service 40, text data obtained through conversion of voicedata received from the electronic whiteboard 10 (step S1906). Inresponse, the decision-making assistance service 40 transmitsdecision-making assistance data to the electronic whiteboard 10 via thevoice conversion service 30.

At this time, the decision-making assistance service 40 determines, forexample, a text represented by the text data received from the voiceconversion service 30 includes a confidential word or a similar word. Ifthe decision-making assistance service 40 determines that the textrepresented by the text data includes a confidential word or a similarword, the decision-making assistance service 40 transmitsdecision-making assistance data indicating that confidential informationis included to the electronic whiteboard 10 via the voice conversionservice 30.

The decision-making-assistance-data determiner 110 of the electronicwhiteboard 10 determines whether the decision-making assistance datareceived via the communicator 104 indicates that confidentialinformation is included (step S1907).

If it is determined in step S1907 that the decision-making assistancedata indicates that confidential information is included, as in stepS1709 above, the display control 102 displays the screen G500illustrated in FIG. 18 (step S1908).

On the other hand, if it is determined in step S1907 that thedecision-making assistance data indicates that confidential informationis not included, the display control 102 displays decision-makingassistance information (step S1909).

In the above manner, with the conference assistance system 1 accordingto this embodiment, for example, a user of the electronic whiteboard 10depresses the “YES” button G510 on the screen G500 illustrated in FIG.18 to refrain from future transmission of contents of the conference(voice data). On the other hand, with the conference assistance system 1according to this embodiment, for example, a user of the electronicwhiteboard 10 depresses the “NO” button G520 on the screen G500illustrated in FIG. 18 to continue transmission of contents of theconference (voice data).

As described above, with the conference assistance system 1 according tothis embodiment, based on the decision-making assistance data receivedfrom the decision-making assistance service 40, a screen for selectingwhether transmission of contents of the conference is to be suspended isdisplayed on the electronic whiteboard 10. Thus, with the conferenceassistance system 1 according to this embodiment, for example, if theagenda of the conference is changed to confidential matter during theconference, transmission of contents of the conference can be changed tobe suspended during the conference.

Next, a fourth embodiment will be described. In the fourth embodiment,in the electronic whiteboard 10, handwritten-character data and voicedata are converted into text data, and it is determined whether a textrepresented by the text data includes confidential information.

<Functional Configuration>

First, a functional configuration of the conference assistance system 1according to this embodiment will be described with reference to FIG.20. FIG. 20 illustrates an exemplary functional configuration of theelectronic whiteboard 10 according to this embodiment. Note thatfunctional components that are substantially the same as those in thefirst embodiment will not be repeatedly described.

As illustrated in FIG. 20, the electronic whiteboard 10 according tothis embodiment includes a voice conversion processor 111, a characterconversion processor 112, and a confidentiality determiner 113. Each ofthese functional units is implemented by the CPU 17 performing processesin accordance with one or more programs installed in the electronicwhiteboard 10.

In addition, the electronic whiteboard 10 according to this embodimentdoes not include the transmission permission determiner 108. Theelectronic whiteboard 10 according to this embodiment further includes aconfidential-word-information memory 114. This memory can be implementedby using, for example, the HDD 18. Note that the memory may beimplemented by using a memory device or the like connected to theelectronic whiteboard 10 via a network.

The voice conversion processor 111 converts voice data generated by thevoice-data generation processor 107 into text data by using, forexample, the voice recognition technology or the like.

The character conversion processor 112 converts handwritten-characterdata generated by the handwritten-character-data generation processor105 into text data by using, for example, the OCR technology or thelike.

Referring to confidential-word information stored in theconfidential-word-information memory 114, the confidentiality determiner113 determines whether text data obtained through conversion performedby the voice conversion processor 111 and the character conversionprocessor 112 includes a confidential word or a similar word.

The confidential-word-information memory 114 stores confidential-wordinformation. Here, the confidential-word information stored in theconfidential-word-information memory 114 will be described withreference to FIG. 21. FIG. 21 illustrates exemplary confidential-wordinformation stored in the confidential-word-information memory 114.

As illustrated in FIG. 21, in the confidential-word information storedin the confidential-word-information memory 114, each confidential wordis associated with similar words that are similar to or related to theconfidential word.

Note that the confidential words are registered, for example, by a userof the electronic whiteboard 10. In addition, for example, on the basisof the confidential words registered by a user or the like, the similarwords are determined by the electronic whiteboard 10 or a system that isconnected to the electronic whiteboard 10 via a network or the like byusing the of AI technology or an IA.

<Details of Processes>

Next, details of processes performed by the conference assistance system1 according to this embodiment will be described.

First, a case will be described with reference to FIG. 22 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, a participant or the like at the conference inputs contents ofthe conference by handwriting. FIG. 22 is a sequence illustrating anexemplary process of inputting a handwritten character according to thisembodiment. Note that step S2201 and step S2202 are substantially thesame as step S1701 and step S1702 in FIG. 17, respectively, andtherefore description thereof will be omitted.

The electronic whiteboard 10 according to this embodiment performs stepS2203 through step S2209 every time an operation of inputting ahandwritten character is performed. Note that step S2203 and step S2204are substantially the same as step S1703 and S1704 in FIG. 17,respectively, and therefore description thereof will be omitted.

Following step S2204, the character conversion processor 112 of theelectronic whiteboard 10 converts the handwritten-character datagenerated in step S2203 into text data (step S2205).

Subsequently, referring to confidential-word information stored in theconfidential-word-information memory 114, the confidentiality determiner113 of the electronic whiteboard 10 determines whether at least one of aconfidential word and a similar word is included in a text representedby the text data (step S2206).

If it is determined in step S2206 that none of the confidential word andthe similar word is included, the communicator 104 of the electronicwhiteboard 10 transmits the text data to the decision-making assistanceservice 40 (step S2207). In response, the decision-making assistanceservice 40 transmits decision-making assistance data to the electronicwhiteboard 10.

Upon reception of the decision-making assistance data via thecommunicator 104, the display control 102 of the electronic whiteboard10 displays decision-making assistance information (step S2208).

On the other hand, if it is determined in step S2206 that at least oneof the confidential word and the similar word is included, the displaycontrol 102 of the electronic whiteboard 10 displays the screen G500 forselecting suspension of transmission of contents of the conference, suchas the screen illustrated in FIG. 18 (step S2209).

In the above manner, the electronic whiteboard 10 according to thisembodiment converts handwritten-character data into text data anddetermines whether a text represented by the text data includes aconfidential word or the like. Thus, the electronic whiteboard 10according to this embodiment can transmit only text data including theconfidential word or the like to the decision-making assistance service40.

Next, a case will be described with reference to FIG. 23 in which, at aconference or the like that uses the electronic whiteboard 10, forexample, voice of a speech of a participant or the like at theconference collected as contents of the conference is input. FIG. 23 isa sequence illustrating an exemplary process of inputting voiceaccording to this embodiment. Note that step S2301 and step S2302 aresubstantially the same as step S1901 and step S1902 in FIG. 19,respectively, and therefore description thereof will be omitted.

The electronic whiteboard 10 according to this embodiment performs stepS2303 through step S2309 every time voice is collected by the soundcollecting device 19. Note that step S2303 is substantially the same asstep S1903 in FIG. 19, and therefore description thereof will beomitted.

Following step S2304, the voice conversion processor 111 of theelectronic whiteboard 10 converts voice data generated in step S2303into text data (step S2304).

Subsequently, the display control 102 of the electronic whiteboard 10displays a text represented by the text data obtained through conversionin step S2304 above (step S2305).

Subsequently, referring to confidential-word information stored in theconfidential-word-information memory 114, the confidentiality determiner113 of the electronic whiteboard 10 determines whether at least one of aconfidential word and a similar word is included in the text representedby the text data (step S2306).

If it is determined in step S2306 that none of the confidential word andthe similar word is included, the communicator 104 of the electronicwhiteboard 10 transmits the text data to the decision-making assistanceservice 40 (step S2307). In response, the decision-making assistanceservice 40 transmits decision-making assistance data to the electronicwhiteboard 10.

Upon reception of the decision-making assistance data via thecommunicator 104, the display control 102 of the electronic whiteboard10 displays decision-making assistance information (step S2308).

On the other hand, if it is determined in step S2306 that at least oneof the confidential word and the similar word is included, the displaycontrol 102 of the electronic whiteboard 10 displays the screen G500 forselecting suspension of transmission of contents of the conference, suchas the screen illustrated in FIG. 18 (step S2309).

In the above manner, the electronic whiteboard 10 according to thisembodiment converts voice data into text data and determines whether atext represented by the text data includes a confidential word or thelike. Thus, the electronic whiteboard 10 according to this embodimentcan transmit only text data including the confidential word or the liketo the decision-making assistance service 40.

Although the above first to fourth embodiments have described exemplarycases in which handwritten characters and voice are input to theelectronic whiteboard 10, the present invention is not limited to thesecases. Each of the above embodiments is applicable to an apparatus otherthan the electronic whiteboard 10.

For example, each of the above embodiments is applicable to a scannerthat generates image data by scanning a document. In this case, in thesecond embodiment above for example, before a scanner scans a document,a user can select whether image data is to be transmitted to thedecision-making assistance service 40. In addition, in the fourthembodiment above for example, if image data generated by a scannerincludes a confidential word or the like, a user can select whether theimage data is to be transmitted to the decision-making assistanceservice 40.

Furthermore, for example, each of the above embodiments is applicable toa projector that projects an image based on image data. In this case, inthe second embodiment above for example, a user can select whether imagedata of an image to be projected is to be transmitted to thedecision-making assistance service 40. In addition, in the fourthembodiment above for example, if image data of an image to be projectedincludes a confidential word or the like, a user can select whether theimage data is to be transmitted to the decision-making assistanceservice 40. More specifically, in this embodiment, text data isextracted from the image data to be projected, while the image data maybe provided in the form of an electronic file, for example, from aterminal operated by the user.

Furthermore, for example, each of the above embodiments is applicable toa videoconference terminal that transmits and receives image data andvoice data to and from another terminal. In this case, in the secondembodiment above for example, a user can select whether image data andvoice data to be transmitted and received to and from a plurality ofterminals are to be transmitted to the decision-making assistanceservice 40. In addition, in the fourth embodiment above for example, ifimage data and voice data to be transmitted and received to and from aplurality of terminals include a confidential word or the like, a usercan select whether the image data and voice data are to be transmittedto the decision-making assistance service 40. As described above, textdata is extracted from the image data to be displayed, while the imagedata may be provided in the form of an electronic file, for example,from a terminal operated by the user. Alternatively, any writing made bythe user, for example, on the electronic whiteboard 10 or any material(such as paper) on a table may be captured using a camera of thevideoconference terminal, for display. In such case, the image databeing displayed corresponds to data of a drawing made by the user on theelectronic whiteboard 10 or any writing material.

In addition, although each of the above embodiments has described a casein which the electronic whiteboard 10 is connected to thedecision-making assistance service 40 through the network N1, thepresent invention is not limited to this case. That is, for example, thedecision-making assistance service 40 may be included in the conferenceassistance system 1.

Furthermore, for example, the electronic whiteboard 10 may have afunction of implementing a service provided by the decision-makingassistance service 40 (i.e., decision-making assistance function). Inthis case, the transmission permission determiner 108 of the electronicwhiteboard 10 determines whether decision-making is to be assisted byimplementing a decision-making assistance function by using text datarepresenting voice data, handwritten-character data, or the like. Inthis case, the decision-making assistance function of the electronicwhiteboard 10 generates assistance information based on the text data.

Further, in some embodiments, the conference assistance system includesa plurality of computing devices, e.g., a server cluster, that areconfigured to communicate with each other over any type ofcommunications link, including a network, a shared memory, etc. tocollectively perform the processes disclosed herein.

In any one of the above-described embodiments, the decision-makingassistance service 40 for assisting a user to make a decision is used asa system that utilizes conference information that is obtained duringthe conference using an electronic apparatus, such as the electronicwhiteboard 10, projector, videoconference terminal, etc. Alternatively,any service capable of assisting a user by providing conference-relatedinformation, may be used other than the decision-making assistanceservice 40. Examples of such service include, but not limited to,organization management service, operation planning service, reportingservice, etc.

Further, contents of a conference, such as discussion or agenda, are notlimited to the one described above. Examples of the contents ofconference include, but not limited to, reporting, scheduling,brainstorming, problem or issue identifying, coaching one or moremembers related to a project, or any other type of conferences ormeetings that are usually held to achieve a goal to be shared bymultiple users, for example.

The above-described embodiments are illustrative and do not limit thepresent invention. Thus, numerous additional modifications andvariations are possible in light of the above teachings. For example,elements and/or features of different illustrative embodiments may becombined with each other and/or substituted for each other within thescope of the present invention.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. Processing circuitryincludes a programmed processor, as a processor includes circuitry. Aprocessing circuit also includes devices such as an application specificintegrated circuit (ASIC), digital signal processor (DSP), fieldprogrammable gate array (FPGA), and conventional circuit componentsarranged to perform the recited functions.

1. An information processing apparatus comprising: a network interfaceconfigured to communicate with a conference assistance server through anetwork; one or more input devices configured to receive input of voiceor writing of at least one meeting participant during a meeting; andprocessing circuitry configured to, receive an instruction indicatingwhether to transmit data related to the input of the voice or thewriting to the conference assistance server, transmit the data, via thenetwork interface, to the conference assistance server, in response tothe instruction indicating to transmit the data, receive, via thenetwork interface, assistance information associated with the data fromthe conference assistance server, the assistance information includinginformation relevant to the meeting, and control a display device todisplay the assistance information together with the data related to theinput of the voice or the writing such that the assistance informationdisplayed on the display device assists the at least one meetingparticipant.